Papers with semantic plausibility
The Dog the Cat Chased Stumped the Model: Measuring When Language Models Abandon Structure for Shortcuts (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can explain quantum mechanics and write sophisticated code, yet fail to parse sentences like "The cat that the mouse feared chased meowed" |
| Approach: | They propose a framework to distinguish structural understanding from semantic pattern matching . they use a set of 9,720 comprehension questions on center-embedded sentences . |
| Outcome: | a new framework shows that models lose performance when they abandon structural analysis for semantic associations. |
Modeling Semantic Plausibility by Injecting World Knowledge (N18-2)
Copied to clipboard
| Challenge: | Existing models for semantic plausibility are based on distributional data, but injecting knowledge about entity properties provides a substantial performance boost. |
| Approach: | They propose to inject manually elicited knowledge about entity properties into a dataset to improve plausibility models. |
| Outcome: | The proposed dataset is a great testbed for semantic plausibility models . it shows that injection of knowledge about entity properties improves performance . |
Thinking in Schemas: Robust Syllogistic Reasoning in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | syllogistic reasoning models often mistake what sounds true for what is formally valid . content effect is a limitation of sluggish reasoning, which can lead to invalid conclusions . eisape et al., 2024: a key open problem for formal inference in natural language. |
| Approach: | They propose a schema-guided framework that disentangles semantic plausibility from logical validity. |
| Outcome: | The proposed framework outperforms existing frameworks while reducing CE. |